Rigorous Evaluation of COTS Middleware Technology

M iddleware refers to a broad class of software infrastructure technologies that use high-level abstractions to simplify construction of distributed systems. Over the past decade, the adoption of commercial off-the-shelf middleware products across the software industry has gathered significant momentum. COTS products make it possible to deploy application components in high-performance, scalable, and fault-tolerant environments. Most major organizations rely on some form of COTS middleware components to integrate new Internet applications with existing core business systems and to manage evolving business processes. While COTS middleware products demonstrably solve many significant problems, their adoption and use is by no means straightforward. Because these technologies typically provide business-critical services, selecting the appropriate middleware platform is essential. System and, ultimately, business failures—caused by large-scale COTS middleware products that didn’t work as anticipated—litter the industry. Competition among products that appear to offer identical services further complicates the COTS middleware selection process, especially when competing products provide different implementations of standards-based technologies, such as Corba (www.omg.org) and Sun Microsystems’ Java 2 Enterprise Edition (J2EE, http://java.sun.com/j2ee/). The Middleware Technology Evaluation (MTE) project rigorously evaluates COTS middleware components and technologies. As the “MTE’s Fundamental Principles” sidebar describes, the project bases its evaluations on three basic principles. Project output includes technical evaluation reports of key COTS middleware packages and a repeatable methodology for evaluating these products. By providing reliable, comprehensive product assessments, the MTE project helps organizations reduce the up-front costs they incur in selecting suitable COTS middleware products. This in turn should significantly reduce the downstream costs of system development by eliminating most risks associated with complex technology selection and procurement early in the process.